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Verbal Coding as an Instructional Strategy In Improving 
Pupil Performance on Standardized Measures of Reading Comprehension* 

Recent writers in education (Finder^ 1969; SlmonSt 1971) h^ve mentioned 
the Importance of distinguishing between the product and process involved in 
reading comprehension. The psychological processes used in reading compre- 
hension are covert mental acts and hence not amenable to direct observation. 
On the other hand, proJuct operations are observations made ^bout a subject's 
behavior once the process has occurred. Thus^ research efforts regarding 
the processes of reading comprehension have been of a post hoc nature. That 
is, subjects are directed to read passages and then answer various questions 
about them. From the responses received the investigator then maKes inferences 
with respect to the psychological processes the subjects might have used. 

Simons cites and evaluates seven methodolical approaches that have been 
used by researchers to shed light on both the products and processes of 
reading comprehension. These include: listing of skills, measurement of 
pupil performance on standardized tests, factor analysis techniques, 
correlational studies, readability research^ study of readers' introspective 
reports, construction of psychological models, and psycholinguistic research* 
With the exception of psycholinguistic research efforts which look at surface 
and deep structure in passages, Simons contends that very little knowledge is 
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available to explain the psychological processes utilized in reading 
comprehension. This '*sad state of the art*' is particularly perplexing to 
classroom teachers, especially those teaching beyond the primary grades. 
Reading instruction beyond word recognition stages, has typically stressed 
the importance of acquiring reading comprehension skills. Practice material 
in workbooks and basal readers are used by teachers and pupils that presc 
reading comprehension skills in a sequential and ordered format. Pup:i 
practice with these materials in order to learn the skills of reading 
comprehension. Teachers frequently complain that the materials designed 
to accommodate growth in comprehension skills are ineffective, i.e., pupils 
seem unable to transfer the learning that presumably occurred from the 
practice exercises to other content or skill areas. This difficulty is 
further complicated by the myriad of standardized tests that are available 
to Gelineate pupil strengths and weaknesses in reading comprehensions. 

Of the previous research methods mentioned, the introspective or 
thinking aloud technique, on the surface, appears to be amenable to direct 
classroom practice to provide pupils with an instructional strategy for 
improving reading comprehension. In introspective studies, subjects are 
asked to verbalize about how they answered particular comprehension 
questions. For example. Smith (1967) used an introspective technique with 
eighth grade subjects to obtain data on the success of subjects ability to 
answer comprehension questions* She also attempted to gather data to 
describe the processes used by the subjects in reading for specific purposes. 
She found that good readers differed significantly from poor readers in 
their ability to read for details and general impressions, make adjustments 
to specific purposes for reading, and hold in mind longer their purposes 
O „r reading. 



Introspective or verbalizing techniques employed in reading compre- 
hension research are highly similar to verbal coding research efforts in 
dtfflning human problem solving. One could also make the case that the 
Inferred psychological processes Involved in reading comprehension, as 
defined thus far by investigators, seem highly similar to the processes 
defined as taking place during problem solving activities. Writers in 
cognitive psychology have devoted considerable time and energy in defining 
and describing the hypothetical processes involved in human problem solving* 
The research literature is replete with models that describe human thinking 
and strategies or rules governing problem solving activities, (de Groot, 
1961), (Gagne', 1964), (Johnson, 1964), (Newell, Shaw, and Simon, 1958), 
and (Simon and Newell, 1970). 

Host researchers have described problem solving procesises as pro- 
ceeding tnrough various distinct but overlapping stages. For example, an 
individual I) initially perceives a statement of a problem; i.e., a 
situation is recognized as having "problem-like" properties, 2) defines a 
problem by distinguishing unique or essential features, 3) searches for, 
either systematically or randomly, and formulates hypotheses for the 
solution to the problem, and 4) attempts to verify the solution to the 
problem. In addition to the above stages, D'Zurllla and Goldfrled (1968), 
insist that a predisposition or an orientation must exist as a set or 
attitude, within an Individual to facilitate problem solving activities. 
Such a predisposition may be an internal or medlational tendency on the 
part of an individual to be aware of problem situations. In addition. 
Individuals will need a set of cognitive operations or strategies for 
solving problems. Consequently, the individual is able to recognize and 
accept problematic conditions when they occur and react either Impulsively 



4 

or passively in the face of the problem situations. 

Typically, the delineation of stages or strategies for problem 
solving have been the product of research efforts where subjects have been 
observed solving algebra word problems, playing chess or differentiating 
among geometric patterns. The experimental situations were designed to 
determine the subjects reactions to unique or novel situations. 

Two differing methodological techniques have been experir^entally 
utilized by writers to gain insight into how subjects arrive it solutions 
to problems. Paige and Simon (1966), a^iter analyzing hum<in behavior data, 
refined a computer program as a research tool to simulate human information 
processing strategies. Th<^ omputer program, STUDENT, was capable of 
solving algebra word probUms and, as such, attempted to simulate processes 
that subjects conce bly used with the same algebra word problems, 
de Groot (19S5) u^ introspective experimentation, namely verbalizing, to 
describe cognitivt strategies subjects used in various problem solving 
settings. Subject ^ were directed to nhlnk aloud in 1) traditional problem 
solving situations, 2) by reproducing chess positions, and 3) by providing 
definitions of known concepts. Johnson (1964) utilized the thinking-aloud 
procedure in analyzing protocols from subjects who were asked to hypothesize 
concerning properties which differentiated one set of patterns from another. 
The analysis of the verbal transcripts yielded an information-processing 
model Judged to be successful in the simulation of problem solving behavior 
of subjects on a variety of problems. 

Writers have rigorously defended the use of the verbal coding technique 
as a viable experimental technique for collecting problem solving data. Its 
primary purpose is to provide the researcher with information about the 
cognitive processes the subject uses while solving a. problem, de Groot (1965, 



p. 21) contends that such experimental procedures yield data that 
conceivably are a linear sequence o£ operations that are actively carried 
out by the subject according to a system of linkings governed by laws. 
The analysis of such data then, should yield a classification of human 
mental operations to cover every method, trick (sic), heuristic that may 
^e conceived o€ as problem solving. 

The purpose of the present study was to determine if verbalizing, 
while attemptitig to seek the solution to a typical reading comprehension 
question, would enhance pupil performance on a later standardized measure 
of reading comprehension. The pupils were provided with an audio-taped 
message that demonstrated how to solve a particular problem. The examiner 
on the taped message read the stimulus passage and then proceeded to read 
the comprehension questions. Each of the foils was read and the exauiiner, 
referring to the previously read passage, would state reasons as to why 
each foil could be correct or incorrect. The subjects were told to model 
the taped message. Each set of comprehension questions was preceeded with 
taped instructions. T\e subjects were also provided an opportunity to 
practice modeling the examiner prior to attempting the items in the experiment. 
All subjects in the study practiced verbalizing and audio taped protocols 
were secured from each subject. Subjects were randomly assigned to 
practice groups to determine the effects of varying the amount of practice 
time, use of differing stimulus materials, and reinforcing comments on the 
part of the examiner, 

METHOD 

SUBJECTS 

61 seventh grade pupils from an urban public school were selected for 
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the study. In order to minimize reading problems it was decided to Include 
only pupils with average scores on a standardized measure of reading 
comprehensioa in the study. Therefore » pupils with grade equivalent scores 



of 7.0 or greater on the sub 'asts of vocabulary and reading comprehension 
on the Form IM^ Level D Gates-MacGlnitie Reading Teit, administered in the 
fall 1971, were included. 51 pupils wore randomly assigned to eight treat- 
ment groups and 10 pupils were assigned to a control group. 



All subjects were administered Form ^M, Level E of the Gates*MacGinitie 
Reading Test as a pre test measure. This measure was later used as a 
covariate in the analysis of the data to determine mean performance 
differences. The treatment subjects were then randomly assigned to eight 
practice groups » where the three treatments, previously described, were 
presented in differing orders. 

Each treatment subject then met individually with the examiner and 
pract£ced with audio-taped instructions describing the experiment* The 
instructions were utilized to demonstrate how to verbalize and the subjects 
were asked to "think out loud" when answering the comprehension questions. 
Subjects were directed to model the examiner. After reviewing the directions, 
the ^ubjects were allowed to practice reading a sample Item before proceeding 
with the treatment. If errors occurred the examiner corrected the subject 
before proceeding. Each item, printed with primary type on a separate 
page, was presented individually in a loose leaf binder. Subjects were 



directed to read the.stlmrlus passage and then answer the questions presented 
on the lower portion of the page. 

The 10 Ss of the control group completed the pre and post test sessions 



PROCEDURES 




treatment group Ss. They received no instruction in verbalizing. 
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FIGURE I 

Treatment Procedures Utilized in Verbal Coding Study 
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A Like materials 

^ Reinforcing commenCs 

^ Daily practice (five consecutive days) 

^ One practice (in five day experimental period) 

^ Non reinforcement (no comment from examiner) 

^ Unlike materials 

The treatments shown in Figure 1 were as follows: 

I) Stimulus materials used in the study. 

Subjects in groups 1 through 4 were Instructed to verbalize 
using like stimulus materials (L). These consisted of rep-^ 
resentative items chosen from the Gates^MacGlnltie Reading 
Test, Form 2M, Level E. The test yields four separate scores. 
Speed, Accui;acy (both scores from the same sub test). 
Vocabulary, and Comprehension. Subjects in groups 5 through 
8 verbalized with materials that differed in both content 
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and format froa the Cates-MacGinitle itetac. Thete Iteias 
were constructed and used in a previous study (Schrelncr, 
1968) ana were defined: 1) verbal reasoning, 2) cause 
mod eltect^ 3) locating the main Idea of a paragraph, and 
4) speed of noting details. The unlike Items are labelled 
PL in Figure 1. Each subject practiced verbalizing with 
one sanple Item of each type before the responses were 
tape recorded. Three Items of each type were presented 
In each practice session. 

2) Reinforcing comments from the examiner: 

Subjects in groups 1, 2, 5 and 6 were told by the examiner 
after verbalizing each item that the answer chosen was 
correct and the procedure used was satisfactory. IC the 
subject chose an incorrect response, the examiner directed 
the subject, in a reassuring manner, to the correct answer. 
The groups receiving reinforcing comments are labelled R 
in Figure 1. 

Subjects in groups 3, 4, 7 and 8 received no reinforcement 
or feedback from the examiner. Subjects received instruction 
on the verbalizing technique and the examiner, except for 
directing the subject to the next item, remained mute. The 
non-reinforcing treatment is labelled NR in Figure 1. 

3) Amount of practice time provided for verbal coding: 
Subjects in groups 1, 3, 3 and 7 practiced verbalizing with 
the stimulus materials for thirty minutes a day for five 
consecutive days. Subjects in all groups met individually 
with the examiner; protocols were obtained for all subjects 

ERLC 
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on audlo-tapc* The &ubjcct& who practiced daily are labelled 
D in Figure 1 • 

Subjects in groups 2, ^, 6^ and 8 practiced verbalizing with 
the Bti^nulus ma erials once during the cxpcrltncntal period. 
Practiced in these groups were staggered throughout the 
experimental period so that an equal number of subjects 
practiced on day one through day five. The groups receiving 
one practice verbalizing are labelled 0 in Figure 1. 

ANALYSIS OF THE DATA 

A one-way analysis of covariance to determine the extent of treatment 
effects, was performed on each of the adjusted mean scores for the four 
separate sub tests of Form 3M, Level E of the Cates^MacCinitic Reading Test. 
The adjusted mean scores of eight treatment groups and the control group 
were included in these analyses. DunnctC*s t^ statistic (Winer, 1962, p. 89) 
was used to compare each treatment with the control condition to determine 
significant mean differences. Finally, a three-way analysis of covariance 
was performed on each of the adjusted mean scores for the four separate 
sub tescs achieved by the eight treatment groups. The experimental variables 
of interest in this study were: 1) like materials vs. unlike materials, 
2) five daily practices vs. one practice session, and 3) reinforcement vs. 
non-reinforcement. The covariate in each apalysi£> was the mean score of 
each of the four sub tests of Form IM, Level E of uhe Gates-MacGinitle 
Reading Test which served as a pre test measure. 

RESULTS 

F^ratios generated by the one-way analysis of covariance failed to 
Q iveal significant treatment effects favoring the verbalizing groups over 

ERIC 



10 



th« control group on the sub testf of speed, accuracy, and vocabulary. 
Hovever, a treatoent effect difference (F « 1.94, df; 8/61, p < .07) was 
observed cn the separate sub test of reading comprehension. 

(Insert Table 1 here) 

Raltvant pre test, unadjusted post test, and adjusted post test 
atani are presented In Table 1. Ths post test data were used in determin* 
Log significant differences between treatment and control conditions* 32 
separate calculations were tnade using Dunnett's t, statistics, as a 
posteriori test. Nona of the differences observed approached statistical 
significance, (p. < .05) It should be noted, however, that the use of 
Duanittt's £ statistic imposes a penalty when obtaining the approximate 
error term in the denominator of the equation for calculating significant 
mean differences* Hence, while none of the differences was statistically 
different, an examination of Table 1 reveals that 24 of 32 unadjusted post 
test scores obtained by the treatment groups were equal to or greater than 
similar scores achieved by those members of the control group. In addition, 
among those Ss that verbalised, 23 or 28 unadjusted post test mean scores 
were greater than the mean pre test measures on the same sub tests. These 
results would appear to lend support to the notion that verbalizing appears 
to be an appropriate instructional technique for producing higher scores on 
a standardized measure of reading comprehension* 
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TABLE 2 

Three-Way Analysis of Covariance Using Adjusted 
Mean Scores Between Eight Treatment Groups on Comprehension Si jst 



Source 


df 


SS 


MS 


F 


p- value 


Practice 




118.88 


118.88 


3.95 


.05 


Reinforcement 




40.49 


40.49 


1.34 


.25 


Practice x Reinforcement 




173.57 


173.57 


5.76 


.02 


Materials 




5.41 


5.41 


.18 


.67 


Practice x Materials 




1.65 


1.65 


.06 


.82 


Reinforcement x Materials 




39.24 


39.24 


1.30 


.26 


Reinforcement x Practice 
X Materials 




11.12 


11.12 


.37 


.55 


Adjusted Error 


42 


1264.54 


30.11 







P-ratlo6 generated by the three-way analysis of covariance between 
treatment groups failed to reveal significant inain effect or interaction 
effect differences on the adjusted mean scores on the sub tests of speed, 
accuracy, and vocabulary. However, significant main effect differences 
(p ^ •OS) were found when comparing mean post test differences in favor of 
those groups that received daily practice, A significant practice x 
reinforcement interaction effect (p < ,02) were also observed. The results 
of the three-way analysis of covariance between the adjusted means for the 
eighu treatment groups are shown in Table 2. 
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FIGURE 2 

Practice x Reinforcement Interaction Effects 
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34.0 
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Reinforcement 



Daily practice 
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Non reinforcement 



Figure 2 graphically represents the practice x reinforcement inter-* 
action effect observed from the three-way analysis of covariance performed 
on adjusted mean scores on the comprehension sub test. It was concluded 
that reinforcing comments from the examiner to the Ss appeared to facilate 
approximately equal scores despite the amount of practice times involved. 
(X = 36.8, daily practice vs. 36.7, one practice) However, non-reinforcement 
on the part of the examiner appeared to have a deleterious effect on the 
scores of those Ss who received but one practice session. On the other hand, 
daily practice, despite reinforcing or non-reinforcing comments resulted in 
significantly higher mean post test scores on the reading comprehension sub- 
test. 

A further examination of Table 1 revealed that 7 of the 8 treatment 
groups had lower mean post test scores than pre-test scores on the sub-test 
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of comprehension. One possible explanation of this result may be related 

> 

to the format of the comprehension sub test. Comprehension was measured 
on this sub-test, using a modified cloze procedure. Subjects were required 
to read sentences in paragraphs with selected words deleted. From five 
alternative words presented below the text, the subjects were to choose the 
appropriate response. The subjects completed the remainder of the passage 
in this manner. Since time limits were a factor on the test, Ss who took 
time to verbalize may have been undully penalized for their intervening 
efforts. 

DISCUSSION 

The results of this study would appear to substantiate the ure of 
verbalizing or thinking aloud techniques as an instructional procedure in 
the classroom. Additionally, the results indicate that several practice 
sessions in verbalizing would be necessary to enhance growth in comprehension 
performance. Finally, the practice efforts should be accompanied by re- 
inforcing or reassuring comments on the part of the teacher. 

It should be noted that the increased scores on the measures of 
speed, accuracy, and vocabulary by members of the treatment groups could be 
^related to experimental conditions. Perhaps the pupils had a clearly under- 
stood purpose or set for reading the stimulus materials. Consequently, the 
pupils would more readily analyze the content of the written passages. 
For example, subjects tended to substitute synonyms found in the question 
portion of the test for words they recalled from the passages. Thus, they 
appeared to look for ^similar words before deciding that a response was 
correct. It would appear that verbalizing may cause readers to be analytic 
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in identifying syntactic relationships among words and phrases in printed 
passages. This condition would tend to explain the lower post test scores 
for the treatment subjects on the sub test of comprehension when Ss read 
connected discourse from which words were deleted. 

It is shown in Table 2 that no significant main effect or interaction 
effect differences were found between treatment groups practicing with 
differing materials. One group practiced with an equivalent form of the 
pre and post test measure while the other group used materials that differed 
completely in format and content. This apparent lack of transfer of learning 
from the treatment condition would have some instructional implications for 
the improvement of reading comprehension skills. Logically one would expect 
greater mean achievement from the group that practiced with the equivalent" 
form of the pre and post test measure* The results would seem to indicate 
that the methods rather than the specific materials used were responsible for 
Improved post test scores. This result would indicate that caution should be 
exercised when using materials or standardized reading tests that purport to 
teach or test specific reading comprehension skills. 

Additional study needs to^ be made regarding the lasting effects of 
Instruction in verbalizing as an asset in improving comprehension. If such 
a technique proves to be useful for the improvement of reading comprehension, 
a follow-up study after a lapse of time would be appropriate. It should also 
be noted that this study would be classified as product research by Simons 
criteria. Presently, a transformational-generative analysis of the taped 
protocols is beiag- conducted to determine if meaning can be attached to pupils 
performance during tl^e study. Hopefully, such an analysis will shed some light 
on the psycho -linguistic processes used during reading comprehension activities. 
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Finally, it should be noted that the results of this study appear to be 
applicable to those pupils whose reading comprehension scores were greater 
than grade seven. The results would not appear appropriate for pupils who 
demonstrate decoding difficulties. 
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